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@ Different kinds of video noise filters are known. Normally such filters need a certain amount of memory, e.g. 
field memories, or are not suited for every noise level or are not adaptive to the local structure of the picture. 
The inventive solution is glot>ally and locally adaptive. A noisy input signal x is filtered with a restoration filter of 
median type to generate a filtered input signal y. The sum of the absolute differences between filtered and 
unfiltered signal is calculated for each position of a sliding window within the input signal, representing a local 
estimate of the noise, and is combined with a global measure of the input signal noise to compute two 
coefficients a and b which are respectively applied to the unfiltered and filtered signal to generate the output 
signal z = aV + b*y- 

Advantageously different kind of filters operate in parallel, whereby the kind of filter elected Is locally adapted to 
the picture activity. 
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The present invention relates to a method and to an apparatus for noise reduction. 
Background 

5 Different kinds of video noise filters are known. M. Unser, "Improved restoration of noisy Images by 
adaptive least-squares post-filtering". Signal Processing 20 (1990), pp. 3-14, gives an example. Normally 
such filters need a certain amount of memory, e.g. field memories, or are not suited for every noise level or 
are not adaptive to the local structure of the picture. 

TO Invention 

It is one object of the invention to disclose a method for noise reduction, which does need only simple 
hardware and takes into account different noise levels and picture structures. This object is reached by the 
. method disclosed in claim 1 . . , , 

75 it is a further object of the invention to disclose an apparatus which utilises the inventive method. This 
object is reached by the apparatus disclosed in claim 9. 

For the inventive noise reduction the main requirements listed below are considered: 

* field processing i.e. without field memory; 

* few memories and simple hardware realisation; 
20 * processing adaptive to any level of noise; 

* processing adaptive to the local structure of the picture, 

i.e. detecting plain areas, edges and fine details so as to preserve the relevant Information of the picture. 
Different filters could t>e used: 
1 . linear low pass filter with different coefficients; 
25 2. median filter; 

3. linear filter in the direction of the edges (directional filter); 

4. directional median fiKer; 

5. weighted averages of filters 1 , 2, 3 and 4; 

6. filtering with one of the filters 1 , 2, 3 and 4 with fixed threshold and local measure of picture activity by 
30 the Laplacian. 

These filter types have the following properties: 

* median fitters are more suitable to remove noise in plain areas than linear filters especially in the case 
of high firequency noise of high amplitude; 

* median filters preserve edges and linear filters do not; 

35 * directional filters are very efficient for preserving edges and picture accuracy while filtering slightly but 
efficiently enough; 

* median filters completely lose the fine textures (of a zone plate test picture, for example) and the only 
way to preserve them Is to use a directional filter; 

* weighted averages of filters give good results but the weighting coefficients are not easy to adapt 
40 locally without correct noise estimation and can lead to hard switching and to the Introduction of 

artefacts; 

* thresholds and local nteasures of activity give good results when adapted to the picture but they have 
to be combined with noise estimation on the whole picture. 

The inventive solution Is globally and locally adaptive. The implemented filter is derived from that 

45 proposed by Unser (cited at)ove) and works as follows: 

The noisy input signal x is filtered with a restoration filter of median type to generate a filtered input signal 
y. The sum of the absolute differences between filtered and unfiltered signal is cateulated for each position 
of a sliding window within the input signal, representing a local estimate of the noise, and Is combined with 
a global measure of the input signal noise to compute by least squares regression two coefficients a and b 

50 which are respectively applied to the unfiltered and filtered signal to generate the output signal z - aV + 

Advantageously the only required prior Information Is the noise variance a^, assuming that the original 
signal ii Is degraded by additive stationary noise n. The size of the local noise estimation window can be 
different from the size of the window used by the restoration filter, wherefore they can be adjusted 
55 independentiy for optimal performance. 

The weighting coefficients a and b are made optimal in the sense that they are minimising the quadratic 
error between the output signal z and the noise free original signal u. It Is true that the original signal is 
not known at a receiver, but global noise statistics can be calculated at the receiver side (e.g. described in 

2 



EP 0 588 181 A1 



EP-A-92400785) and used for that optimisation. Preferably a+b = 1. so that no bias is introduced by the 
filtering. This solution usually performs slightly worse, but is rather simple to implement, no computational 
complexity need be added. 

If the filtering tends to degrade the signal, a predominant weight will be given to the unfiltered signal x. 
Conversely, when the sum of differences is small and close to a reference value corresponding to the 
residue of the noise alone, the weight will be shifted to the filtered signal. 

The choice of the kind of restoration filter does heavily influence the merit of the inventive noise 
reduction. Therefore, advantageously different kind of filters can operate in parallel, whereby the kind of 
filter elected has to be locally adapted to the picture activity. That branch which results in a minimum error 
between the filtered and the unfiltered signal will be selected. 

In principle the inventive method is suited for noise reduction, in which for estimating a local noise value 
within a sliding window of a picture the quadratic error between an unfiltered noisy input signal and the 
filtered input signal is computed and used for calculating a weighted average of the filtered and unfiltered 
Jnpij^ signal as outp^ for calculating said weighted average aisp a globial noise value is, 

used and for computing said local noise value pixel difference signals of said window are cafculated. 

Advantageous additional embodiments of the inventive method are resulting from the respective 
dependent claims. 

In principle the Inventive apparatus includes: 

- a first filter, a second filter and a third filter having a common Input signal and a different kind of 
characteristic; 

- a first control circuit controlling the weighting values of a first multiplier operating on said input signal 
and a second multiplier operating on the output signal of said first filter, whereby the outputs of these 
multipliers are combined in first adding means; 

- a second control circuit controlling the weighting values of a third multiplier operating on the output 
signal of said first filter and a fourth multiplier operating on the output signal of said second filter, 
whereby the outputs of these multipliers are combined In second adding means; 

- a third control circuit controlling the weighting values of a fiftii multiplier operating on the output signal 
of said second filter and a sixth multiplier operating on the output signal of said third filter, whereby 
the outputs of these multipliers are combined in third adding means; 

- first subtraction means which feed the difference signal between said Input signal and the output 
signal of said first filter to said first control circuit; 

- second subtraction means which feed the difference signal between said input signal and the output 
signal of said second filter to said second control circuit; 

- third subtraction means which feed the difference signal between the output signal of said first filter 
and the output signal of said second filter to said second control circuit; 

- fourth subtraction means which feed the difference signal between said Input signal and the output 
signal of said third filter to said third control circuit; 

- fifth subtraction means which feed the difference signal between the output signal of said second filter 
and the output signal of said third filter to said third control circuit; 

- further subtraction means which feed the respective difference signals t>etween said input signal and 
the output signals of said adding means to a control and mixing circuit which selects an output signal 
of said adding means according to the minimum error derived from these difference signals and 
according to a global noise variance value. 

whereby for each of the branches said weighting values are derived from error signals calculated In the 
respective control circuit fi'om said difference signals and from said global noise variance value. 

Advantageous additional embodiments of the Inventive apparatus are resulting from the respective 
dependent claim. 

Drawings 

Preferred embodiments of the invention are descril)ed with reference to the accompanying drawings, 
which show in: 

Rg. 1 first embodiment of an inventive noise reduction circuit; 
Rg. 2 filter-related directions In the picture; 
Rg. 3 window for noise filtering; 

Rg. 4 second embodiment of an inventive noise reduction circuit; 
Rg. 5 correction terms for the comparison of the filters errors. 
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Preferred embodiments 

In Fig. 1 noise n is added in a channel. e.g. a TV channel, to a noise free original signal u, e.g. a TV 
signal. The resulting noisy input signal x of the inventive apparatus, e.g. in a TV receiver or a VCR, is 

5 filtered with a restoration filter MF of median type to generate a filtered input signal y. The difference A 
between signal x and signal y is calculated in a subtracter SUB and fed to a control circuit CON which 
forms from A the sum of the absolute differences between filtered and unfiltered input signal for each 
position of a sliding window within the input signal x. representing a local estimate of the noise. 
The global noise statistics can be calculated in a noise measurement circuit NM. The local noise estimate is 

10 combined in control circuit CON with the global measure of the input signal noise to compute by least 
squares regression two coefficients a and b which are respectively applied to the unfiltered and filtered 
signal to generate the output signal z = a*x + b*y- This can be done by multiplying signal x in a first 
multiplier MUL1 by factor a and by multiplying signal y in a second multiplier MUL2 by factor b. The 
outputs of MUL1 and MUL2 arj? added in adder ADDO, resulting in a noise-reduced output signal z. 

15 Because the kmd ot filter MF influences the resulting noise reduction and picture quality, advanta- 
geously different filter types can operate in parallel. Then the problem Is. how to select the output of the 
best-fitting filter. In Rg. 4 three filters DMF, MED and AV of different kind are operating, whereby weighted 
averages of each two consecutive filter outputs are calculated. 

Each average is obtained as in Rg. 1 by minimising the squared error between the output signal z and 
20 the noise-free signal u. A respective filtered output signal Zi, i=0 to 2. is selected by examining the 
respective errors: the selected branch corresponds then to the minimum error. This method allows 
switching between filters depending on the statistical properties of the local filtered and unfiltered pictures. 
The three restoration filters are: 
DMF - directional median filter; 
25 MED - median filter; 
AV - average. 

The sliding window 30 depicted in Rg. 3 has a size of 5 pixels by 3 lines. As the processing is not on a 
fi^ame basis but on a field (F1; F2) basis, tiiis size corresponds to a region of 5 by 5 pixels for the interlaced 
picture (lines of field F1). 
30 The directional median filter DMF works as follows: 

Let Xji, j = 0 to 2. 1 = 0 to 4. be the pixels taken into account by window 30. The tested filter directions in Rg. 
2 are called di, i = 0 to 5. The related gradients are respectively estimated by: 

go = jxoo-xi2j + jx2+-xi2j; 

35 gi = |X01-Xi2| + |X23-Xi2|; 
g2 = |X02-Xl2| + |X22-Xl2|; 
ga = |X03-X12| + |X21-X12|; 
g* = |X04-Xl2| + tX20-Xi2|; 

gs = 1/2*(jxi0-Xl2j + jxil-Xi2j + jxi3-Xi2j + jxi4-Xi2j). 

40 

The selected direction is given by the smallest gradient. The respective output signals yi of filter DMF 
are median values: 

mo = median (xoo. xi2, X24^); 
45 mi = median (xoi, xi2, X23); „ , 
m2 = median (xo2. X12. X22); 
ma = median (xo3, X12, X21); 
m* = median (xo*. xt 2, X2o): 
nr>5 = median (xi 1 , xt 2 . xi 3). 

50 

If two or more of these directions result in a minimum gradient, the average of the respective signals is 
taken to generate the output y^ . Such multiple minimum gradients appear at less than 1% of pixels. 
The output y2 of filter MED is: 

55 
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72 = median (xji) 
0<i<4 
0<j<2 

5 

The output ya of filter AV is: 

^0 73 = * S (xji) 

-"■^ 0<i<4 
0<j<2 

75 In Rg. 4 the circuits ADD, DMF, SUB2, CONO. MUL1 . MUL2 and ADDO correspond to the circuits ADD, MF, 
SUB. CON, MUL1. MUL2 and ADD of Rg. 1. The input of filter DMF is also connected to a second filter 
MED and to a third filter AV and can also be connected to an additional filter AF which forms the output 
signal yo from the noisy input signal x. The input signal x is also fed to a first, a third and a fifth subtracter 
SUB1. SUB3 and SUBS. The output signal yi, y2 and ys. respectively, of fitter DMF, MED and AV, 

20 respectively, is subtracted in the subtracter SUB1 , SUB2 and SUBS, respectively fronn the input signal x. 
The outputs Do, Di , and D2, respectively, of these subtracters are each connected to a respective control 
circuit CONO. C0N1 and C0N2. The difference signal Ao. Ai and A2. respectively, calculated in a second, 
a fourth and a sixth subtracter SUB2, SUB4 and SUB6. respectively, between signals yo ( = x) and yi, yi 
and Y2, and y2 and ys, respectively, are also fed to the according control circuit CONO. C0N1 and C0N2. 

25 If filter AF is omitted, SUB1 can also be omitted t)ecause Db = Ao. 

The first control circuit CONO controls the coefficient ao of a first multiplier MUL1 which multiplies signal yo 
and the coefficient bo of a second multiplier MUI-2 which multiplies signal yi. The outputs of these 
multipliers are added in adder ADDO, forming output signal zo. 

The second control circuit CON1 controls the coefficient ai of a third multiplier MUL3 which multiplies 
30 signal yi and the coefficient bi of a fourth multiplier MUL4 which multiplies signal y2. The outputs of these 
multipliers are added in adder ADD1 . forming output signal zi . 

The third control circuit C0N2 controls the coefficient a2 of a fifth multiplier MUL5 which multiplies signal y2 
and the coefficient b2 of a sixth multiplier MUL6 which multiplies signal ys. The outputs of these multipliers 
are added in adder ADD2. forming output signal Z2. 
35 In a seventh, eighth and ninth subtracter SUB7, SUB8 and SUB9, respectively, the difference signals 
between the according output signals zo, zi and Z2 and the input signal x (error on window) are calculated. 
These subtracter output signals are used in a fourth control and mixing circuit COM for connecting one of 
the output signals zo. zi and Z2 with output z, according to the minimum error. 

For calculating the minimum error in the four control circuits, the known noise variance o^. as descrit>ed for 
40 Rg. 1 . is used. Hard or soft switching can be made in control circuit COM. 

The quadratic errors to be minimised in CONO. C0N1 and C0N2 are for i = 0 to 2: 

= 1/N * E (z. - 
R ^ 

= 1/Nr* S^..^,^..^^ 

where Nr is the total number of pixels in the region R which conresponds to the sliding window 30. The 
notation Sy.v is used to designate a sum of squares over the region R and is defined as: 

55 

Su,v = I u(k:,l)v(k,l) , 
R 
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where k and I are elements of R and u and v stand either for x, Yj. zi or u. u is not known so that Syt.u 
cannot be computed in such a way. Instead, use is made of an a prior knowledge of the noise statistics. 
It is assumed for simplicity that ai and bi are constant over the region R. Then (equation 1): 

= ai^SYi^Yi +t)i^Sy.^^,y.^^ ^Six.fi +2a.b . Sy . ,y . -2a.Sy.,M 



The constraint is ai + bi - 1. The minimisation with this constraint leads to the following system of 
equations: 

aiSyi,yi + biSy,,y,+i + X, = Sy,.u 
ajSyi.yi+1 + biSyi+i,yi+i + Xj = Syi+i,u 
aj + bi = 1, 

where X[ is a Lagrange multiplier. 
The solution is: 

aj = ( Sy,.u -Syi+i^ -Syi,yi+i +Syi+i,yi+i)/Ai as = ( Syi^u -Syi+i,u -Syi,yi+i +Syi+i.yi+i)/Ai 

b, = (-Syi.u +Syi+t^ -Sy,.y,+i +Sy,.y, yA, 

where Ai= Syi.yi -2Syi,yi+i +Syj+i.yi+i = Syryi+i,yryi+i 

Because u = x - n, 

Syi,u = Syi.x -Syi,n = Sx.yi -SUi\n -Sn^n = Sx.yi -pjo^ where for each filter Fj: Ui is the filtered noise free 
signal, nf Is the filtered noise, = E(n2(k. I)) is the known noise variance and pj = E(n(k, l)ni'(k. I)) is the 
residual noise correlation coefficient after filtering with F| (i e. p = 1: no filtering; p = 0: full filtering, no noise 
left). 

The assumption of stationary noise allows here easy estimation of these coefficients p\. But if they are 
known for another kind of noise. It is possible to extend the filter to less restrictive conditions. 
Using these coefficients, aj and bi can be estimated to: 

ai ^ (Sx,yi-Sx.yi+i -Syt,yi+, +Syi+i,yi+i -pja^ +pi+ia2)/Ai 
bj = (-Sx.yj +Sx.yi+i -Syi.yj+i +Syi.yi +pia2 -pj+ia^yAj 

or 

aj = (Sx-yi+i.yryi+i -"^(pi -pi+i))/Syryi+i.yryit.i . 
bi = (Syrx,yryi+i +a2(pi-p,+i))/Syryi+i.yryi+i 

Therefore aj and b| are known once the correlation term Sx-yi+i.yryi+i has been computed within the 
control circuits CONO. C0N1 and C0N2 for the current position of tfie window 30. The output signals Do. Di 
and D2. respectively, of the according subtracters SUB1, SUB2 and SUB3 are used for computing these 
conrelation terms. For good stability of the system it is required that Z| is always between yi and yi+i, i.e. 0 
^a;^ 1. 

For selecting in control and mixing circuit COM the optimum filter branch, it is possible to derive from 
the values of a and b the quadratic errors: 

NR€i = SzrU,ZrU = Szi,Zj -2SZi.u +SUM 

where 
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Sa.a = Sx-n.u = Sx,u -Sn.u = Sx^ -Sx.n = Sx,x -Sn.n -Su,n = Sx^ -a^ 
and 

5 

Sz,.u = Saiyi + biYi+i^u = aiSyi.u +btSyi+i.u = aiSx.yi -aipio^ +biSx,y,+i -bipi+io^ = Sx.Z| -a^- 
(ajpi +biPi+i). 

Therefore 

10 

Nr€, = Sz,,Z| -2Sx.z, +2a2(aiPi +bipi+i) +Sx.x-o2 = Szrx,zrx +a2(2aip, +2bipi+i-1). 

The correction term ej = a2(2ajpj +2bjpj+r1) can be explained by the fact that the more a given filter 
smooths the picture, the higher the term Szrx,zrx will be. Therefore, before the terms generated by fitters 
75 of different efficiencies can be compared, they must be compensated by a specific kind of function. 

The graph of this function is given In Rg. 5 to illustrate this effect when using the current filter type (AV. 
MED, DMF. AF) parameters. The correction term Oj is shown in relation to coefficient a|. 

The filter AV of type 'average* is well suited for plain areas. The two median filters DMF and MED 
preserve the edges and a good resolution. 
20 To improve the contrast, enhancement processing can be implemented after the noise reduction process- 
ing. 

ff the window 30 Is enlarged, also other picture degradations can be reduced, for example blurs and stripes. 
Advantageously due to field processing no blurs are Introduced on moving objects. 

In order to reduce the complexity and the costs of the hardware Implementation it Is possible to modify 
25 the described filters in the following way: 

* the size of the window for error estimation can be reduced to 5 pixels by 1 line: therefore no 
additional line memories will be needed (only 2 line memories for the input filters); 

* filter DMF can be a directional median fitter with only four directions instead of six: do , d2 > d4 and 6s ; 

* fitter MED can be a median with a window of three pixels by three lines; 

30 * limitation of the number of bits for the computation of the weighting coefficients (4 bits, for example). 
These simplifications have the following Influences on the processed pictures: 

* if the size of the estimation window is smaller, the simplified filter will tend to filter more than the non 
simplified one, particularly when there are local horizontal line structures, because no care is taken of 
the previous and following lines even if they are very different in their local structure; 

35 * simplifications of the input filters do not create too many degradations on the output signal since the 
tittering Is adaptive and always the b>est solution between the three outputs zb, zi and Z2 is chosen; 

* the reduction of the number of btts of the weighting coefficients creates no significant loss of 
accuracy: 1 or 2 untts for an amplitude range of 256. 

The big advantage of this new fitter Is to allow with very few memories and simple hardware implementation 
40 a noise fitted ng that is adaptive simuttaneously locally and globally. 

Because of this double ability of adaptation the fittering is optimised even with simple fitters at the input. 

The inverrtive noise reduction can be used in any video or audio processing devices, e.g. TV receivers 
and VCR's. 

45 Claims 

1. Method for noise reduction, in which for estimating a k>cal noise value within a sliding window (30) of a 
picture the quadratic error between an unfiftered noisy input signal (x) and the filtered (MF) input signal 
(y) Is computed and used for calculating (CON) a weighted (a, b) average of the filtered and unfittered 
50 input signal as output signal (z). characterised in that for calculating (CON; CONO, C0N1, C0N2) said 
weighted (a, b; ao, bo, ai, bi, a?, bz) average also a global noise value (NM, o^) is used, whereby for 
computing said local noise value pixel difference signals (A; Ac, Ai A2) of said window are calculated 
(CON; CONO. C0N1, C0N2). 

55 Z Method according to claim 1, characterised in that said fittering is carried out in two or more 
branches of different fitter characteristics (DMF, MED, AV) each having a weighted average output 
signal (zo, zi , Z2) derived from the fittered input signals (yo, yi ; yi , y2; y2, ys) of each two consecutive 
branches, whereby for computing said local noise value in each of said branches additionally respective 
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pixel difference signals between each two consecutive of said filtered input signals (yi, y2; ya, ya) are 
used and for obtaining the final output signal (z) that branch output signal (zo, zi , 22) is selected (COM) 
which has a mininnum error compared to said input signal (x). 

5 3. Method according to claim 2, characterised in that when comparing said minimum error, a correction 
term related to the respective filter characteristic is added to each of the branch errors. 

4. Method according to any of claims 1 to 3, characterised in that the sum of the two weighting factors 
(ao. l>o; ai . bi ; a2, ba) for said weighted average/s is 1. 

10 

5- Method according to claim 1 and 4. characterised in that the one weighting factor (a) related to said 
unfiltered input signal (x) of the two (a. b) weighting factors for said weighted average is in the range 0 
to 1. 

15 a Method according to claims 4 and 5 and to claims 2 or 3, characterised in that the respective one (ao. 
ai , 32) of each two (ao, bo; ai . bi ; aa. ba) weighting factors for said weighted average is In the range 0 
tor 

7. Method according to any of claims 2 to 4 and 6. characterised in that at minimum two of the following 
20 filter characteristics are used in said branches: directional median filter (DMF). median filter (MED), 
average (AV), whereby for said directional median filter at minimum four directions (do-ds) are 
evaluated in said sliding window (30). 

a Method according to any of claims 1 to 7. characterised in that said sliding window (30) has a size of 
25 five pixels by one to five lines when related to a frame basis. 

9. Apparatus for a n^ethod according to any of claims 2 to 4 and 6 to 8. including: 

- a first filter (DMF^, a second filter (MED) and a third filter (AV) having a common input signal (x) 
and a different kind of characteristic; 

30 - a first control circuit (CONO) controlling the weighting values (a©, bo) of a first multiplier (MUL1) 

operating on said input signal (x) and a second multiplier (MUL2) operating on the output signal 
(yi ) of said first filter, whereby the outputs of these multipliers are combined in first adding means 
(ADDO); 

- a second control circuit (C0N1) controlling the weighting values (ai. bi) of a third multiplier 
35 (MUL3) operating on the output signal (yi) of said first filter and a fourth multiplier (MUL4) 

operating on the output signal (ya) of said second filter, whereby tiie outputs of tiiese multipliers 
are combined in second adding means (ADD1); 

- a third control circuit (C0N2) controlling tfie weighting values (32, bz) of a fiftti multiplier (MUL5) 
operating on the output signal (ya) of said second filter and a sixtti multiplier {MUL6) operating on 

40 the output signal (ya) of said third filter, whereby the outputs of these multipliers are combined in 

third adding means (ADD2); 

- first subtraction means (SUB2) which feed the difference signal (Ao) between said input signal (x) 
and the output signal (yi ) of said first filter to said first control circuit (CONO); 

- second subtraction means (SUB3) which feed the difference signal (Di) between said input signal 
, ^ (x) and the output signal (y2) of said second filter to said second control circuit (C0N1); 

- third subtraction means (SUB4) which feed the difference signal (Ai ) between the output signal 
(yi) of said first filter and the output signal (y2) of said second fifter to said second control circuit 
(C0N1); 

- fourth subtraction means (SUBS) which feed ttie difference signal (D2) between said input signal 
50 (x) and the output signal (ya) of said ttiird filter to said third control circuit (C0N2); 

- fifth subtraction means (SUB6) which feed the difference signal (A2) between the output signal 
(y2) of said second filter and the output signal (ya) of said third filter to said third control circuit 
(C0N2); 

- ftjrther subtraction means (SUB7, SUB8. SUB9) which feed the respective difference signals 
55 between said input signal (x) and the output signals of said adding means (ADDO, ADD1. ADD2) 

to a control and mixing circuit (COM) which selects an output signal (zo. zi, Z2) of said adding 
means according to the minimum error derived firom these difference signals and according to a 
global noise variance value (o^. 
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whereby for each of the branches said weighting values are derived from error signals calculated in the 
respective control circuit from said difference signals and from said global noise variance value (a^). 

10. Apparatus according to claim 9, characterised in that said first filter (DMF) is a directional median 
5 filter wherein at minimum four directions (do-ds) are evaluated and said second filter (MED) Is a median 
filter and said third filter (AV) is an averaging filter. 
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